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15 September 2014 by Robert Sesek 

The NSA TURBULENCE program was first revealed in 2007 by the Baltimore Sun for being over- 
budget and poorly-managed Will. Many documents from the Snowden trove from the past two 
years reference parts of this program, but no coherent story or overview of how these programs 
operate together has been described. This is an attempt to do so, using leaked and declassified 
documents. 

At the time the Sun reported on TURBULENCE in 2007, it had an annual cost approaching $500 

million dollars, and it was comprised of nine smaller programs this article covers five of 
those that have been revealed to date. The predecessor program to TURBULENCE was called 

TRAILBLAZER, and it was shut down for cost overruns and mismanagement ^1. The 
TURBULENCE leadership chose to use "several smaller programs [as] a way to hedge bets on 

uncertain technology" unlike TRAILBLAZER which was a monolithic project. While 
TURBULENCE was reported to have had a rocky start, it is now clearly operational and central to 
NSA's 21st century SIGINT mission. 

Passive Collection: TURMOIL 

The TURMOIL program is NSA's global passive SIGINT apparatus. TURMOIL is a "high-speed 
passive collection systems intercept [for] foreign target satellite, microwave, and cable 

communications as they transit the globe" Presumably all this collection is related to 
Internet data (DNI), rather than other forms of SIGINT. 

NSA collects using TURMOIL through a variety of sources. One of those is the RAMPART-A 
program, which provides foreign third-party "collection against long-haul international leased 

communications through special access initiatives with world-wide SIGINT partnerships" ^. 
The Internet links that are tapped through RAM-A have "access to over 3 Terabits per second" of 
data, and "every country code in the world is seen at one or more RAMPART-A collection 

accesses" &\ In other words, friendly communications companies provide access to their 
backbone links, and those "partners work the fiber projects under the cover of an overt Comsat 

[communications satellite] effort" 
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It is likely that the other RAMPART projects, such as "RAMPART-M for access to undersea 

cables" and "RAMPART-T for land-based cables, in cooperation with CIA" [ ^ ] contribute to the 
TURMOIL collection platform. One other specific program that contributes to TURMOIL is the 
MUSCULAR program, in which the NSA tapped the fiber optic links of Google and Yahoo's 

datacenters MUSCULAR and another cable-tapping program called INCENSER are part of 
the WINDSTOP project, rather than RAMPART, access to which occurs through a second party, 

reported to be GCHQ There are likely several other cable-tapping programs that contribute 
to TURMOIL. 

A unique aspect of the TURMOIL program is that it operates on the packet level, rather than 

operating on full sessions Communications on the Internet, such as an email or webpage, 
are broken down into tiny pieces called packets when they are transmitted, allowing each small 
piece of data to take the fastest route possible to its destination. When all the packets arrive at 
the destination, they are reassembled and made whole again. This reassembly, and grouping 
related email messages and webpages, is what the NSA refers to as "sessionized" data, since the 
data are full sessions rather than broken-down packets. 
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Because TURMOIL operates on the packet level, it can choose to handle certain types of traffic 
with special processing. From leaked documents, it is clear that there are at least three unique 
types of traffic that are handled at the TURMOIL level. 

The first two are part of a system called APEX ^ and handle VPN and VoIP traffic. In the case of 
VPN traffic, a system called HAMMERSTEIN identifies the traffic and sends the metadata to a 

database called TOYGRIPPE. The TOYGRIPPE database is a "repository of VPN endpoints" [l ^ ] that 
is used by targeting officers to determine if that computer should be a target for further 

exploitation The TURMOIL VPN module also looks up the IP information in a database called 
KEYCARD to determine if the target should be tasked for targeted SIGINT collection or to recover 
the VPN key. Of special note is that this VPN traffic passes through a system called "TE-VPN PIQ 
Blade." PIQ almost certainly refers to the PICARESQUE ECI marking, which is associated with 

BULLRUN The BULLRUN program is NSA's effort to weaken and exploit encryption that 
protects digial SIGINT, whether by finding bugs in cryptographic algorithms or by manipulating 
standards bodies or companies into weakening encryption tools. It is safe to assume that PIQ is a 
compartment that contains the details of a cryptologic attack against a specific VPN technology 
(or technologies), which the NSA/GCHQ either found or paid for. 
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Source: The Intercept ^ 



VoIP traffic is processed through a module in TURMOIL called HAMMERCHANT, which processes 
the digital telephony through CONVEYANCE, for storage in NUCLEON - just like a regular 

telephone call In this way, NSA analysts can listen on audio communications regardless of 
whether it was carried over the PSTN or VoIP. 



The third module is the apparent ability to do decryption of encrypted Internet traffic Only 
one slide shows TURMOIL interacting with a hardware-accelerated decryption element, but this 
is likely performing attacks against weak Internet cryptographic standards. This information 
broke in the stories about the BULLRUN cryptographic exploitation programs, but no strong 
connection was made between BULLRUN and TURMOIL in previous reporting about these 
programs. It is unclear if NSA can decrypt (weakly enciphered) Internet traffic in real-time, but 
based on the diagram's similarities to HAMMERCHANT and HAMMERSTEIN, that seems 
possible. 
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Active Collection: TURBINE 

The TURMOIL system sifts through Internet traffic, identifying that which provides useful 
foreign intelligence information. This system feeds several databases (more on that below), but 
one of TURMOIL'S main counterparts is TURBINE. The TURBINE system is deeply integrated with 
the QUANTUMTHEORY Computer Network Exploitation (CNE) techniques used by the NSA. 
When TURMOIL detects packets on the Internet containing a selector (i.e. some identifying 
information) targeted by NSA, it "tips" the TURBINE system. This "tipping" triggers a program 
in the TURBINE system to attempt to deploy an active exploit, using the tasked 
QUANTUMTHEORY attack method, on the target computer. There are several individual 

QUANTUMTHEORY attack methods from which an officer can select ^— \ the end goal for all is to 
infect the targeted computer such that NSA can maintain access for exfiltrating information. 

The TURBINE system "provides centralized automated command/control of a large network of 

active implants" kJJ. Because this system sits between the passive collection system, TURMOIL, 
and the active attack mechanisms of QUANTUM, it is described as having the ability to "talk to 

active and passive sensors/shooters" One set of slides describes these elements quite 
succinctly: "Detect: TURMOIL passive sensors detect traffic & tip TURBINE command/control. 
Decide: TURBINE mission logic constructs response & forwards to TAO [Tailored Access 

Operations] node" [ ^ ] . Once the traffic is forwarded to TAO, the QUANTUMTHEORY attack 
commences. 



QUANTUMTHEORY operates by racing packets across the Internet: when a user attempts to 
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reach the destination webserver. The webserver then responds with its own stream of packets to 
deliver the reply. TURMOIL, by virtue of sitting in the middle of the Internet backbone, can 
detect when a targeted user's packets are being sent across the Internet. When TURMOIL tips off 
TURBINE, the QUANTUMINSERT attack attempts to reply to the user's request by sending its 
own stream of packets, before the real webserver's packets can be delivered to the user. If this 
happens, the packets sent by QUANTUMINSERT quickly redirect the user to a NSA TAO server, 
which installs the implant. After implantation, the user is redirected back to her original 
destination, with the hope that the implantation was done covertly. If successful, the NSA can 
access any data on the computer and maintain this privileged access. (This is why web browser 
security is serious business). 
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In this diagram, the components of a QUANTUMINSERT attack are displayed. Two separate 
TURMOIL systems (each at a collection site) are integrated with the TURBINE system. It is likely 



that the "MHS" site refers to Menwith Hill Station, one of NSA's main collection facilities, and 
"TEC" refers to another, but unknown, site. The TURBINE svstem also interacts with a database 
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called TRAFFICTHIEF, the purpose of which will be analyzed below. The diagram is also split into 
two sides: low side and high side, referring to the network on which the part runs. The high side 
components run in NSA's classified TOP SECRET NSANet network, to protect itself from 
discovery (CNE is covert action) and to protect the components from outsiders. The low side 
components are responsible for implantation, and are managed by TAO on the public Internet. 
To communicate across the low-high side divide, NSA uses something called a "data diode." Two 
are seen in this diagram as green circles with an arrow, one allowing traffic to go from high-to- 
low and the other for low-to-high. The diode labeled SH is an abbreviation for the codename 

SURPLUSHANGAR it is unknown what the label CDR is an abbreviation for. 

According to NSA documents, TURBINE is quite effective and has implanted thousands of 
computers. However an issue appears to have lessened its efficacy, due to the way the system 
was designed. On all diagrams, TURMOIL is spread across the globe, but TURBINE is centralized 
at NSA's headquarters in Maryland (also known as NSAW, for NSA-Washington). When 
attempting to QUANTUMINSERT a target that is geographically far away from the TURBINE 
system, the latency caused by communicating between the implantation system, TURBINE, and 
TURMOIL took too long and QUANTUM was losing the packet race. To work around this problem, 

NSA came up with QFIRE, which has the goal to "eliminate trans-Atlantic/Pacific latency" ^ by 
moving the mission logic and decisions to be local to the collection site. With this revised design, 
TURMOIL no longer has to tip TURBINE back at NSAW; the TURBINE logic resides in the same 
geographic location as TURMOIL. NSA employees probably still use the same TURBINE interface 
(GUI) to select, target, and task implantation, but the actual mission logic is now replicated 

across the globe to each collection site that is QUANTUM-enabled ^\ 

Integration with XKEYSCORE 

The XKEYSCORE program was widely reported, since it is NSA's and GCHQ's rolling buffer of 

three days worth of full-take Internet traffic ^— \ According to a GCHQ document, the design of 
XKS has evolved to meet the growing traffic demand. The first iteration of the system, called 
"Traditional" used another system called WEALTHYCLUSTER, which "sessionized all the data on 

the link and presented it all to XKS" For higher data rates though, collecting everything was 
too much, so TURMOIL is used to select interesting packets. 

The next design iteration of XKS, called "Stage 2", leveraged TURMOIL'S access to terabytes of 
information flowing over global communication networks. Remember that TURMOIL operates 
only on packets, not full sessions. To feed XKEYSCORE, it only collects 5% of those packets, 

based on the data meeting various criteria including strong selection (e.g. an email or keyword), 
subnet promotion (specific networks), technology promotion (e.g. Tor), and email domains '- 22 \ 
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Once TURMOIL has collected and saved the selected packets, it forwards them to XKEYSCORE, 
which sessionizes the data and makes it searchable to NSA analysts through the XKS interface. 



The most advanced design iteration is called "Deep Dive", and it can sessionize at "10G data 

rates" ^ 21 \ To do this, all data are sessionized on the link, and the interesting data are 
"promoted" (that is, selected for storage) using the GENESIS language. It is likely that the 
XKEYSCORE source code that Jacob Applebaum published, demonstrating promotion of Tor 

traffic, is written in this GENESIS language ^— \ 

Data Storage and TRAFFICTHIEF 

Most Internet metadata are stored in NSA's main database for that information, MARINA. 
However a separate database operated under TURBULENCE called TRAFFICTHIEF also exists and 

integrates with XKEYSCORE TRAFFICTHIEF is described as containing "meta-data from a 

subset of tasked strong-selectors" On various diagrams, TRAFFICTHIEF appears to 

primarily be used by TURBINE and TAO's implantation servers I estimate that this database 
contains the selectors used to target computers and individuals for TAO operations, 
QUANTUMTHEORY and otherwise. It is likely that the reason TRAFFICTHIEF also feeds into 
XKEYSCORE is because if a target is important enough to collect on with active SIGINT, then 
analysts will want data about the target from all SIGINT databases the NSA has, both passive and 
active. 

Cross-Feeding Databases 

As part of cable-tapping programs like RAMPART, WINDSTOP, and Upstream, the NSA takes the 
signal from the fiber optic cable and sends it through several different systems: 





SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL 
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The "Front-End Cable" in this case is part of the communication provider's network. The 
"Switch Controller" is the first piece of NSA's infrastructure, and it is likely a cable splitter. I 
analyze below how this may be a project codenamed TUMULT. It is apparent from this diagram 
that Internet data are sent through two processing systems. In the diagram, "Tipping" data are 
shown to be sent to TURMOIL, as discussed above, which sends metadata to both MARINA (long- 
term metadata repository) and TRAFFICTHIEF (TURBINE tipping metadata repository). It is also 
shown from this diagram that TURMOIL sends full content to the PINWALE database for storage. 

The diagram also shows that full DNI data passes through a pair of systems, WC-2 and DEEP 
DIVE, before sending data to the main content repository PINWALE. It is likely that WC-2 refers 
to WEALTHYCLUSTER-2, a new version of the system discussed above; and DEEP DIVE refers to 
the third design iteration of the XKEYSCORE architecture. Since TURMOIL operates on the packet 
level, it is more suited to identifying metadata quickly and in real-time. For content, NSA uses 
WEALTHYCLUSTER to collect packets and sessionize them, which is a logical step for feeding a 
database that stores full communication content, like PINWALE. It is likely that NSA uses 
WEALTHYCLUSTER-2 and DEEP DIVE to maintain a majority of traffic displayed in XKEYSCORE 

(the stated goal of which is to have a three-day buffer of all Internet traffic), with the targeted 
communications reserved in PINWALE. This contrasts to TURMOIL, the main function of which 
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described above. 

From the cable taps, NSA also filters out telephony data using LOPERS, which is a software- 
based system for processing PSTN data The data extracted by LOPERS is stored in the FASCIA 
database, which is NSA's repository of cell phone metadata (including location, IMEI, IMSI, and 

other unique identifiers) 

It is unclear to what "ACS" refers in the last prong of the processing diagram, but the data goes 
to two separate databases: TWISTEDPATH and BLACKPEARL. It is unknown what data are stored 
in TWISTEDPATH, but BLACKPEARL is used to store "SIGINT session 5-tupel, identified routers, 

routing protocols, SIGINT access points (inferred SIGINT access points)" The BLACKPEARL 

system is associated with TREASUREMAP, a program to map the Internet ^\ Since this is the 
"Survey" prong of the processing diagram, as in survey and map the Internet, it is likely that 
TWISTEDPATH stores related data. 

Active Defense: TUTELAGE 

This component of TURBULENCE is unlike the others in that it faces inward. The goal of the 

TUTELAGE program is to defend Department of Defense networks ^— \ The Wall Street Journal 
reports that the program "detects incoming cyberattacks and allows NSA to block the threat or 

manipulate the attack code" Similarly, the Washington Post reports that the system "has 
the ability to decide how to handle malicious intrusions - to block them or watch them closely to 

better assess the threat" It has also been associated with the BYZANTINE-HADES program, 
which refers to the "concerted effort against Chinese hackers. It now has a new name. Probably 

containing the word 'LEGION'" The TUTELAGE system has been re-purposed to protect 

civilian networks, like the DHS, under the name EINSTEIN-3 Little more is known about the 
details of this program. Media outlets likely did not publish any documents Snowden had about 
this program, because doing so could inflict actual, articulable harm against the U.S. 
Government, if its network defense strategy were made public. 

Small Mention: TUMULT 

Another project under TURBULENCE is TUMULT. It is only described on one slide, in relation to 
QUANTUMTHEORY and TURMOIL. It is referenced in a slide describing the latencies of various 
nodes in the QUANTUMTHEORY stack: 
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The first node is SAS, described as the "Site Access System Frontend & Layer o/i". Stacked on 
top of that is TUMULT, described only as "Demux & Layer 2". Above this layer is TURMOIL, 
described on the slide as "Layer 3 + passive sensor/event detection". If the layers refer to the OSI 
7 Layer Model , which I think is likely, then the SAS frontend deals with the actual physical fiber 
optic cable and is a hardware component. The TUMULT component is also likely a hardware 
module, and if it operates at layer two of OSI, then it refers to the data link layer. It is possible 
that this is a piece of physical hardware that performs the actual traffic splitting of the fiber 
cable, one part of the split transiting as it should, the other being sent to TURMOIL for 
processing. 

The rest of the latency table describes the full flow of QUANTUMTHEORY. After the TURMOIL 
sensor detects a target, it uses ISLANDTRANSPORT, an "Enterprise Message Service" to 
communicate with TURBINE. This is the tipping mentioned above, and ISLANDTRANSPORT is 
likely something similar to Protocol Buffers RPC or Thrift . TURBINE makes the decision on 
whether to implant, probably in consultation with TRAFFICTHIEF as discussed above. The 

request for implantation then crosses the SURPLUSHANGAR diode to traverse the low- 
side/high-side boundary. The covert network that TAO runs is called MIDDLEMAN, and it then 
performs the packet injection to implant the computer program on the target's computer. 
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According to the NSA slide, QUANTUMTHEORY can implant a target in 686 milliseconds. 



Closing Remarks 

In all the documents about the TURBULENCE program, no mention is made of what authorities 
the projects operate under. The NSA does not perform collection unless it believes it has the 
legal authority to do so; programs like PRISM operate under FAA authority, whereas almost all 
foreign intelligence is collected under EO 12333 . It is common in many leaked briefing 
presentations to state the authority under which collection occurs. But such information is 
absent in any TURBULENCE presentations. Given the slide entitled "SSO [Special Source 

Operations]: Today's Cable Program"^ 1 , 1 believe that TURBULENCE, specifically TURMOIL, is a 
collection platform for all kinds of NSA access. The "three access portfolios" displayed on the 
slide cover domestic cooperation, foreign cooperation, and "unilateral" access. From a resources 
management perspective, it would only make sense for NSA to build one large, all- 
encompassing SIGINT platform, TURBULENCE and TURMOIL, and store the data with special 
labels based on the authority under which it was collected. There are repeated references to 

storing FISA data with labels such as RAGTIME And being able to reuse the same platforms 
and systems would simplify analyst access, pool software development resources, and unify 
systems administration tasks. I believe the TURBULENCE project was NSA's program to 
modernize its SIGINT systems for the 21st century, as communication switched from radio and 
telegraph to the Internet. It appears to be wildly successful. 

This page was last revised 15 September 2014; if new information comes out that either further 
corroborates or disproves this analysis, this page will be updated. 
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